AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time	single_prediction_time
	1_Linear	Linear	f1	0.928889	15.97	0.0281
	2_Default_LightGBM	LightGBM	f1	0.962472	15.92	0.0132
	3_Default_Xgboost	Xgboost	f1	0.962306	16.69	0.0187
	4_Default_CatBoost	CatBoost	f1	0.964758	22.71	0.0245
	5_Default_NeuralNetwork	Neural Network	f1	0.933036	25.43	0.0276
	6_Default_RandomForest	Random Forest	f1	0.922414	19.4	0.1371
	11_LightGBM	LightGBM	f1	0.940171	15.79	0.0113
	7_Xgboost	Xgboost	f1	0.949227	16.08	0.0142
	15_CatBoost	CatBoost	f1	0.964444	14.99	0.0196
	19_RandomForest	Random Forest	f1	0.960177	18.01	0.1148
	23_NeuralNetwork	Neural Network	f1	0.941964	15.17	0.0328
	12_LightGBM	LightGBM	f1	0.967177	16.23	0.0114
	8_Xgboost	Xgboost	f1	0.962637	17.13	0.0187
	16_CatBoost	CatBoost	f1	0.962306	16.62	0.0145
	20_RandomForest	Random Forest	f1	0.925054	19.16	0.1358
	24_NeuralNetwork	Neural Network	f1	0.940133	15.97	0.03
	13_LightGBM	LightGBM	f1	0.966741	18.61	0.0121
	9_Xgboost	Xgboost	f1	0.895323	17.22	0.0154
	17_CatBoost	CatBoost	f1	0.966887	16.82	0.0138
	21_RandomForest	Random Forest	f1	0.904959	19.49	0.1574
	25_NeuralNetwork	Neural Network	f1	0.936819	17.51	0.0298
	14_LightGBM	LightGBM	f1	0.966887	19.25	0.0126
	10_Xgboost	Xgboost	f1	0.666667	17.26	0.0156
	18_CatBoost	CatBoost	f1	0.964758	17.67	0.0137
	22_RandomForest	Random Forest	f1	0.918919	19.75	0.1178
	26_NeuralNetwork	Neural Network	f1	0.942982	17.24	0.0277
	12_LightGBM_GoldenFeatures	LightGBM	f1	0.962306	18.34	0.0284
	14_LightGBM_GoldenFeatures	LightGBM	f1	0.964602	19.7	0.0278
	17_CatBoost_GoldenFeatures	CatBoost	f1	0.958242	18.21	0.0346
	12_LightGBM_RandomFeature	LightGBM	f1	0.964602	19.03	0.014
	27_LightGBM	LightGBM	f1	0.965066	19.07	0.0123
	28_LightGBM	LightGBM	f1	0.964602	19.5	0.0119
	29_CatBoost	CatBoost	f1	0.966741	18.16	0.0156
	30_CatBoost	CatBoost	f1	0.969565	18.75	0.0146
	31_Xgboost	Xgboost	f1	0.953846	19.22	0.0131
	32_Xgboost	Xgboost	f1	0.949227	19.15	0.0151
	33_RandomForest	Random Forest	f1	0.960177	20.82	0.145
	34_RandomForest	Random Forest	f1	0.942184	29.97	0.1295
	35_NeuralNetwork	Neural Network	f1	0.913319	23.52	0.0502
	36_NeuralNetwork	Neural Network	f1	0.913319	19.15	0.0301
	37_NeuralNetwork	Neural Network	f1	0.941964	19.5	0.0409
	38_NeuralNetwork	Neural Network	f1	0.9375	20.61	0.0284
	39_RandomForest	Random Forest	f1	0.958425	24.68	0.124
	40_RandomForest	Random Forest	f1	0.915464	25.32	0.2265
	41_CatBoost	CatBoost	f1	0.969565	25.52	0.0169
	42_CatBoost	CatBoost	f1	0.971302	25.03	0.0504
	43_CatBoost	CatBoost	f1	0.964602	24.46	0.0228
	44_CatBoost	CatBoost	f1	0.969027	26.49	0.0231
	45_RandomForest	Random Forest	f1	0.95614	25.63	0.2178
	46_RandomForest	Random Forest	f1	0.953437	28.85	0.1242
	47_NeuralNetwork	Neural Network	f1	0.942222	21.51	0.0517
	48_NeuralNetwork	Neural Network	f1	0.950893	23.37	0.0302
	49_NeuralNetwork	Neural Network	f1	0.941441	20.24	0.0388
	50_NeuralNetwork	Neural Network	f1	0.941964	25.1	0.0379
the best	Ensemble	Ensemble	f1	0.973568	7.54	0.1539

	score	threshold
logloss	0.693206	nan
auc	0.500868	nan
f1	0.666667	0.452935
accuracy	0.502183	0.505075
precision	0.501818	0.505075
recall	1	0.452935
mcc	0.00445767	0.505075

	score	threshold
logloss	0.693206	nan
auc	0.500868	nan
f1	0.547619	0.505075
accuracy	0.502183	0.505075
precision	0.501818	0.505075
recall	0.60262	0.505075
mcc	0.00445767	0.505075

	score	threshold
logloss	0.333978	nan
auc	0.964922	nan
f1	0.940171	0.494684
accuracy	0.938865	0.494684
precision	1	0.863332
recall	1	0.0302798
mcc	0.878567	0.494684

	score	threshold
logloss	0.333978	nan
auc	0.964922	nan
f1	0.940171	0.494684
accuracy	0.938865	0.494684
precision	0.920502	0.494684
recall	0.960699	0.494684
mcc	0.878567	0.494684

	Predicted as 0	Predicted as 1
Labeled as 0	210	19
Labeled as 1	9	220

	score	threshold
logloss	0.217357	nan
auc	0.975277	nan
f1	0.965066	0.484934
accuracy	0.965066	0.484934
precision	1	0.983528
recall	1	6.92688e-05
mcc	0.930131	0.484934

	score	threshold
logloss	0.231695	nan
auc	0.974934	nan
f1	0.96	0.509248
accuracy	0.960699	0.509248
precision	0.99422	0.767094
recall	1	0.0303221
mcc	0.92196	0.509248

	score	threshold
logloss	0.243977	nan
auc	0.975191	nan
f1	0.960352	0.483092
accuracy	0.960699	0.483092
precision	0.993464	0.762315
recall	1	0.000145718
mcc	0.92196	0.522099

	score	threshold
logloss	0.239979	nan
auc	0.979482	nan
f1	0.964444	0.503711
accuracy	0.965066	0.503711
precision	0.994083	0.757501
recall	1	0.00133263
mcc	0.930699	0.503711

	score	threshold
logloss	0.176663	nan
auc	0.977251	nan
f1	0.964758	0.493538
accuracy	0.965066	0.493538
precision	1	0.948169
recall	1	0.00591764
mcc	0.930273	0.493538

	score	threshold
logloss	0.221495	nan
auc	0.9748	nan
f1	0.962472	0.442244
accuracy	0.962882	0.442244
precision	1	0.971919
recall	1	0.00614362
mcc	0.925985	0.442244

	score	threshold
logloss	0.414492	nan
auc	0.980521	nan
f1	0.964444	0.499648
accuracy	0.965066	0.499648
precision	1	0.601474
recall	1	0.0733196
mcc	0.930699	0.499648

	score	threshold
logloss	0.270494	nan
auc	0.979739	nan
f1	0.960352	0.482133
accuracy	0.960699	0.482133
precision	1	0.843889
recall	1	0.0100179
mcc	0.92196	0.50076

	score	threshold
logloss	0.256123	nan
auc	0.984135	nan
f1	0.964758	0.476858
accuracy	0.965066	0.476858
precision	1	0.853906
recall	1	0.0101198
mcc	0.930273	0.476858

	score	threshold
logloss	0.212563	nan
auc	0.982847	nan
f1	0.956332	0.480383
accuracy	0.956332	0.480383
precision	1	0.842453
recall	1	0.0150311
mcc	0.912803	0.523701

	score	threshold
logloss	0.200394	nan
auc	0.985107	nan
f1	0.964758	0.502573
accuracy	0.965066	0.502573
precision	1	0.854009
recall	1	0.00343254
mcc	0.930273	0.502573

	score	threshold
logloss	0.209104	nan
auc	0.980826	nan
f1	0.955947	0.484429
accuracy	0.956332	0.484429
precision	1	0.945715
recall	1	0.0191015
mcc	0.913221	0.510067

	score	threshold
logloss	0.192042	nan
auc	0.974028	nan
f1	0.941176	0.538017
accuracy	0.943231	0.538017
precision	1	0.88594
recall	1	0.00697453
mcc	0.888635	0.538017

feature	Learner_1	Learner_2	Learner_3	Learner_4	Learner_5
PEER_PRESSURE	2.09855	1.97751	2.10596	2.24214	2.11845
ALLERGY	1.71851	1.8115	1.80494	1.6434	1.99987
CHRONIC DISEASE	1.87657	1.63242	1.72145	1.71654	1.80335
YELLOW_FINGERS	1.60769	1.59401	1.57961	1.70178	1.69653
SWALLOWING DIFFICULTY	1.54115	1.39416	1.52935	1.57504	1.50732
COUGHING	1.6269	1.44695	1.61924	1.30983	1.45889
WHEEZING	1.39098	1.33944	1.35351	1.6218	1.23305
ALCOHOL CONSUMING	1.2172	1.51097	1.01974	1.43756	1.6208
FATIGUE	1.22949	1.23666	1.24562	1.05542	1.27239
ANXIETY	0.676465	1.11655	0.820949	1.03973	0.927658
CHEST PAIN	0.451082	0.609851	0.301484	0.389733	0.68452
SMOKING	0.404751	0.352542	0.398707	0.489795	0.317827
SHORTNESS OF BREATH	0.222579	0.360146	0.254631	0.399099	0.113896
AGE	-0.0683001	0.0651653	0.0293103	-0.0196802	0.14053
GENDER	-0.0705105	-0.180772	-0.240995	-0.128562	0.147742
intercept	-4.88958	-5.06286	-4.78165	-5.16035	-5.28215

	score	threshold
logloss	0.24046	nan
auc	0.976879	nan
f1	0.927039	0.504373
accuracy	0.925764	0.504373
precision	1	0.70647
recall	1	0
mcc	0.852048	0.504373

	score	threshold
logloss	0.297998	nan
auc	0.945911	nan
f1	0.904959	0.494248
accuracy	0.899563	0.494248
precision	1	0.796151
recall	1	0.0269788
mcc	0.804328	0.494248

	score	threshold
logloss	0.325746	nan
auc	0.955426	nan
f1	0.918919	0.498894
accuracy	0.914847	0.498894
precision	1	0.8082
recall	0.995633	0
mcc	0.833911	0.498894

	score	threshold
logloss	0.232964	nan
auc	0.976917	nan
f1	0.944321	0.492837
accuracy	0.945415	0.492837
precision	1	0.99338
recall	1	6.41845e-12
mcc	0.891518	0.492837

	score	threshold
logloss	0.192402	nan
auc	0.974066	nan
f1	0.944321	0.509772
accuracy	0.945415	0.509772
precision	1	0.999026
recall	1	0.000268946
mcc	0.891518	0.509772

	score	threshold
logloss	0.244839	nan
auc	0.966877	nan
f1	0.938865	0.505872
accuracy	0.938865	0.505872
precision	1	0.991168
recall	1	4.76261e-06
mcc	0.877863	0.551479

AutoML Leaderboard

AutoML Performance

AutoML Performance Boxplot

Features Importance

Spearman Correlation of Models

Summary of 10_Xgboost

Extreme Gradient Boosting (Xgboost)

Validation

Optimized metric

Training time

Metric details

Metric details with threshold from accuracy metric

Confusion matrix (at threshold=0.505075)

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

Summary of 11_LightGBM

LightGBM

Validation

Optimized metric

Training time

Metric details

Metric details with threshold from accuracy metric

Confusion matrix (at threshold=0.494684)

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

Summary of 12_LightGBM

LightGBM

Validation

Optimized metric

Training time

Metric details

Metric details with threshold from accuracy metric

Confusion matrix (at threshold=0.484934)

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

Summary of 12_LightGBM_GoldenFeatures

LightGBM

Validation

Optimized metric

Training time

Metric details

Metric details with threshold from accuracy metric

Confusion matrix (at threshold=0.509248)

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

Summary of 12_LightGBM_RandomFeature

LightGBM

Validation

	score	threshold
logloss	0.190663	nan
auc	0.979119	nan
f1	0.947598	0.46477
accuracy	0.947598	0.46477
precision	1	0.999892
recall	1	4.76882e-06
mcc	0.895197	0.46477

	score	threshold
logloss	0.146197	nan
auc	0.981894	nan
f1	0.965066	0.502471
accuracy	0.965066	0.502471
precision	1	0.978055
recall	1	0.000414117
mcc	0.930131	0.502471

	score	threshold
logloss	0.185771	nan
auc	0.977575	nan
f1	0.960352	0.468831
accuracy	0.960699	0.468831
precision	1	0.95207
recall	1	0.0219251
mcc	0.92196	0.508432

	score	threshold
logloss	0.274684	nan
auc	0.982857	nan
f1	0.964444	0.50655
accuracy	0.965066	0.50655
precision	0.990826	0.774811
recall	1	0.00625214
mcc	0.930699	0.50655

	score	threshold
logloss	0.214128	nan
auc	0.975992	nan
f1	0.960352	0.491788
accuracy	0.960699	0.491788
precision	0.994709	0.749729
recall	1	0.016086
mcc	0.921538	0.491788

	score	threshold
logloss	0.261866	nan
auc	0.979215	nan
f1	0.965368	0.47603
accuracy	0.965066	0.47603
precision	1	0.895824
recall	1	0.0031013
mcc	0.930273	0.47603

	score	threshold
logloss	0.262015	nan
auc	0.974619	nan
f1	0.951965	0.488125
accuracy	0.951965	0.488125
precision	1	0.925955
recall	1	0.00842946
mcc	0.904068	0.5026

	score	threshold
logloss	0.2715	nan
auc	0.972054	nan
f1	0.947137	0.494361
accuracy	0.947598	0.494361
precision	1	0.928547
recall	1	0.0124766
mcc	0.895333	0.494361

	score	threshold
logloss	0.200104	nan
auc	0.978833	nan
f1	0.955947	0.470352
accuracy	0.956332	0.470352
precision	1	0.902146
recall	1	0.0139463
mcc	0.913221	0.510375

	score	threshold
logloss	0.246522	nan
auc	0.976659	nan
f1	0.940133	0.575554
accuracy	0.941048	0.575554
precision	1	0.762099
recall	1	0.015909
mcc	0.882508	0.575554